In a cache coherent shared disk computer system, one or more persistent disks are shared among a plurality of nodes, where each node contains memory and one or more processors that share the memory. A portion of the memory of each node may be used as a "buffer cache" which temporarily stores disk resident data accessed by the processors of the node.
Because data on the disk is shared among nodes, the system needs to manage the shared data in a way that ensures each processor or device reading from or writing to the shared data does so in a way that preserves the data in a consistent state. Consider a situation where two nodes are executing separate processes that share a data item, where a copy of the data item currently resides in each node. If a first node modifies its copy of the data item and the second node is not notified of the modification, then the second node may supply an outdated version of the data item to its process, causing an error. However, if a resource management system is established that maintains the data item in a way that makes each copy of the data item appear to be a single, "consistent" data item (e.g., by updating or invalidating the data item in the second node in response to the update in the first node), then that data item is said to be in a "consistent" or "coherent" state.
Each buffer cache is managed by a cache coherency manager. The cache coherency manager for a given buffer cache controls access to the buffer cache and maintains the data in one or more buffer caches in a coherent, or consistent state. In addition, each buffer cache can create "locality", which will be explained in greater detail below.
A shared disk computer system is frequently employed in computing environments, such as database systems, where a number of users and processes may require access to a common database that is persistently stored on one or more shared disks.
FIG. 1 depicts a cache coherent shared disk computer system 100. In FIG. 1, a disk 150, comprising two data blocks 152 and 154, is connected to a disk controller 140 by a local bus 145. The disk controller 140, is connected to a first node 190 and a second node 192 by an I/O network 135.
First node 190 comprises a processor 102, a buffer cache 104 and a cache coherency manager 106. Buffer cache 104 has in it a copy of data block 154 (represented as a cached data block 154'). Processor 102, buffer cache 104 and cache coherency manager 106 are interconnected by a local bus 108.
Similarly, second node 192 comprises a processor 112, a buffer cache 114 and a cache coherency manager 116. Buffer cache 114 has in it a copy of data block 154 (represented as a cached data block 154'). Processor 112, buffer cache 114 and cache coherency managers 116 are interconnected by a local bus 118.
The first node 190 and the second node 192 in the cache coherent shared disk computer system depicted in FIG. 1 are interconnected by a system area network 130. For example, system area network 130 interconnects processors 102 and 112, as well as cache coherency managers 106 and 116.
Various configurations may be used to interconnect processor 102 to buffer cache 104 and a cache coherency manager 106 (e.g. local bus 108). Similarly, various configurations may be used to interconnect first node 190 to second node 192 (e.g. system area network 130). Likewise, various configurations may be used to connect first node 190, second node 192 and disk controller 140 (e.g. I/O network 135). The interconnection configurations shown in FIG. 1 are exemplary and are intended to simplify the description of a shared disk computer system.
Locality in a computer system takes a number of different forms, such as spatial locality, temporal locality and processor locality. Spatial locality is said to exist when contemporaneous memory references are likely to access adjacent or nearby memory addresses. Temporal locality is said to exist when a recent memory reference is likely to be accessed again. Further, parallel computing can create another form of locality called processor locality. Processor locality is said to exist when contemporaneous memory references are likely to come from a single multiprocessor (instead of many different ones).
The use of a buffer cache can create locality between the disk 150 and a process initiated in an interconnected processor by increasing the chances that data required by a processor in the future will be located near the processor. Using cache coherency manager 116, a local process initiated on processor 102 can exploit the temporal locality of accesses to cached data block 154' while it is in adjacent buffer cache 104, instead of being delayed by processing and communication latencies that would result from continually re-reading data block 154 from the disk 150.
In FIG. 1, each cache coherency manger maintains a data block from disk 150 in a consistent state by using a cache coherency protocol. The cache coherency protocol ensures that each processor 102 and 112 has access to a similar, or consistent copy of data block 154, even though the cached data block 154' is distributed in multiple buffer caches. For example, cache coherency manager 116 maintains data block 152 in a consistent state while a copy exists in buffer caches 104 and 114. Likewise, cache coherency manager 116 maintains data block 154 in a consistent state while it is distributed in buffer caches 104 and 114.
The cache coherency managers 106 and 116 in the shared disk computer system depicted in FIG. 1 help to create locality between a buffer caches 104 and 114, processors 102 and 112, and a data blocks 152 and 154 in disk 150.